Research Management Review, Volume 21, Number 1 (2016) 


Return on Investment and Grants: 
A Review of Present Understandings 
and Recommendations for Change 

Michael Preuss 

Hanover Research 


Abstract 

The need to understand efficacy and outcomes from grant-funded activity is common to 
funders, the academic community, and the public. Yet, few articles in the research 
administration corpus offer details on and considerations of applying the concept of return 
on investment (ROI) to grant activity. To determine the volume of material available aside 
from publications specific to research administration that considers systematic assessment of 
ROI for grants, a review was undertaken of the periodic literature available on the ProQuest 
database. A Boolean search for "grant" AND "return on investment" produced over 2,700 
results. Following review, 34 of these sources were found to be relevant to a discussion of the 
systematic application of ROI to grant activity. These articles make it clear that interest in ROI 
for grants is not isolated to a few disciplines or areas of professional practice and that two 
categories of use are common for ROI with respect to grants: "econometric calculation" 

(Frank & Nason, 2009, p. 528) and "impact of...activities" (Weiss, 2007, p. 206). A second 
substantial theme in the literature is the misalignment of fiscal return on investment and 
assessment of grant-supported projects. Establishing assessment patterns that consider 
benefits derived is a preferable pattern. While government agencies in a number of countries 
have initiated processes of this type, their foci will not facilitate local, institution-specific 
benefit analysis. Two patterns for measuring and assessing impacts of grant-funded activity 
are recommended: Uttam and Venugopal's "assessment of benefits from sponsored research" 
(2008, p. 57) (developed for the Indian National Chemical Laboratory) and the rubric-based, 
balanced scorecard approach commonly employed in business settings. 
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Rationale 

The desire to understand the efficacy 
and outcome of an investment is both 
common and wise. The need for such 
understanding extends to grant-funded 
activity for all stakeholder categories and 
especially for the funder, the research 
community, and the public (Weiss, 2007). 

Prior to his work on this study, the 
author knew of only two publications on 
the systematic application of return on 
investment (ROI) to grant activity. This 
understanding was formed during a 
research administration literature review on 
the measurement of grant capacity and 
readiness (Preuss, 2015). In an effort to 
determine whether sources addressing 
assessment of the ROI of grants existed in 
the general scholarly literature, a review 
was undertaken of the periodic literature 
available on the ProQuest database. 

Purpose 

The literature review was designed to 
answer three questions: 

■ How has the concept return on 
investment been applied in periodic 
literature with respect to grant-funded 
activity? 

■ Is there a consensus regarding whether 
and how this construct should be 
applied? 


■ Have systems been developed for the 
application of this principle to grant- 
funded projects? 

In addressing these questions, the 
researcher considered all digitally accessible 
sources on the ProQuest database through 
June 2015. 

Definitions 

Return on investment (ROI) is a 
financial measure that has long been 
employed in the business world to monitor 
performance (Wheelen & Hunter, 2004). It is 
a simple calculation. "To calculate ROI, the 
benefit (return) of an investment is divided 
by the cost of the investment; the result is 
expressed as a percentage or a ratio" 
(Investopedia, 2015, para. 1). Mansfield's 
1991 study of research is a pertinent 
example—he calculated the annual rate of 
return for academic research to be 28%, a 
figure current authors have called into 
question (Mcllwain, 2010, p. 683; Mansfield, 
1991). 

The second key concept in the literature 
review, grant-funded activity, was defined 
as an undertaking for which a scope of 
work, timeline, and performance objectives 
have been defined and a sum of money has 
been provided by a third party for 
expenditure only on a particular 
undertaking by an individual or 
organization. 
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Method 

A modification of the PRISMA pattern 
for systematic review (Moher et al.., 2009) 
was employed in developing the 
investigative method and as a guide in 
reporting. The PRISMA pattern consists of 
"a 27-item checklist.. .and a four-phase flow 
diagram" (Liberati et al.., 2009, para. 6), 
detailing the Preferred Reporting Items for 
Systematic reviews and Meta-Analyses. 

This pattern originated as an approach to 
reporting on literature reviews in the health 
sciences and is described in the following 
way by its creators: 

A systematic review attempts to collate 
all empirical evidence that fits pre¬ 
specified eligibility criteria to answer a 
specific research question. It uses 
explicit, systematic methods that are 
selected to minimize bias, thus 
providing reliable findings from which 
conclusions can be drawn and decisions 
made (Liberati et al., 2009, para. 3). 

The review was conducted between late 
2014 and early summer 2015, beginning 
with consideration of every source available 
on the ProQuest database containing the 
word "grant" and the phrase "return on 
investment." 

Data Sources and Selection 

ProQuest classifies sources within 
publication categories. A Boolean search of 


"grant" AND "return on investment" 
produced over 2,700 results. The listing of 
publication dates had a natural break at the 
year 2000. Only one potentially relevant 
source had been published between 1980 
and 1989. A total of 101 articles published in 
the 1990s contained the word "grant" and 
the phrase "return on investment", with a 
total of 35 between 1990 and 1996,16 in 
1997, 24 in 1998, and 26 in 1999. However, 

58 articles published in the year 2000 
contained both terms, with a rapidly 
increasing number each year from that 
point onward. The researcher elected to 
include only articles from 2000 to 2015 in 
order to pare down the number of possible 
sources, to respect the point at which 
interest in the topic seemed to catch hold 
and accelerate, and to place the 
consideration in a fairly recent context. 
Between January 2000 and June 2015, 2,479 
documents were listed across the eight 
types of publications available on ProQuest. 
In descending order, 1,223 scholarly articles, 
435 dissertations or theses, 358 trade journal 
articles, 256 reports, 201 newspaper articles, 
3 conference proceedings, 2 working 
papers, and 1 publication not otherwise 
classified were identified. Of these, 
newspaper articles were excluded from 
consideration. This decision was taken 
because newspapers are popular media 
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rather than scholarly media and therefore 
highly unlikely to report professional 
understandings of the application of ROI to 
grants or to describe systems intended to 
analyze grant activity ROI. The titles and 
abstracts for the remaining 2,278 potential 
sources were read as a means of identifying 
articles relevant to the topic. 

An article was classified as a potentially 
relevant source if it appeared to apply the 
concept return on investment to an individual 
grant project, to grants in general, or as part 
of a systematic consideration of grant 
activity. Examples of terms considered 
indicative of this type of content included: 
assessing, measuring, evaluating, calculating, 
factors, variables, return on investment, ROI, 
grant, foundation funding, government 
funding, and external funding. If an abstract 
was not present, as is frequently the case 
with material published in trade journals, 
the initial pages of the piece were read to 
identify the article's purpose. The 
judgments regarding the relevance of each 
article to the purpose of this review were 
made by one researcher. All source 
decisions were made by the same 
researcher, eliminating potential for inter¬ 
reviewer bias. 

The number of articles considered as 
possible sources, arranged by the 
classification system employed in ProQuest, 


the number of those from each category 
initially thought to be relevant to the 
investigation, and the number ultimately 
judged to be applicable are presented in 
Table 1. 

More sources were thought to be 
relevant in the initial culling than actually 
proved to be applicable. The most frequent 
cause of this was the use of the term grant 
exclusively as a verb rather than as a noun. 
Authors of works considering the concept 
return on investment often used the word 
grant to communicate giving, allowing, 
admitting, permitting, conceding, and other 
possible synonyms, rather than employing 
the term to refer to externally-funded 
projects, as in the present review. There 
were also whole categories of potential 
source material that yielded no relevant 
sources. None of the potential sources from 
the reports, conference proceedings, 
working papers, or "other" category proved 
to be applicable to the purpose of this 
review. A common reason for the 
elimination of these possible sources was 
mention of return on investment in the 
same work as information about grants 
without seeking to provide the details of the 
connection between the two ideas or a 
system for calculating the ROI. Among the 
reports, 216 Federal Register announcements 
were eliminated because they mentioned 
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ROI without describing an approach or 
prescribing a pattern of calculation. 

Data Gathering and Analysis 

The title and abstract, or title and initial 
pages of articles without abstracts, were 
read to identify possible sources. The initial 


sort identified 93 of the 2,278 search results 
as potentially relevant. A closer reading of 
the 93 documents eliminated another 59 of 
these as sources. Table 2 lists the reasons 
these articles were removed from 
consideration. 


Table 1 

Source Material 


Source Type 

# Articles 

Available 

# Articles 

Thought 

Relevant 

# Articles 

Judged 

Applicable 

Scholarly publications 

1,223 

29 

14 

Dissertations/theses 

435 

18 

7 

Trade journal articles 

358 

46 

13 

Reports 

256 

0 

N/A 

Conference proceedings 

3 

0 

N/A 

Working papers 

2 

0 

N/A 

Other 

1 

0 

N/A 

Total 

2,278 

93 

34 


Note: Excluded sources —articles published prior to 2000; newspaper articles 


Table 2 


Winnowing Potential Source Material 


Source Type 

# Articles 

Thought 

Relevant 

# Articles 

Judged 

Relevant 

Reason Sources Classified as Irrelevant 

Scholarly 

publications 

29 

14 

•Grant activity was described but the concept of ROI was applied to a 
different topic that was also discussed. 

•While financial or other outcomes were discussed, no attempt was made to 
associate them with the concept of ROI. 

Dissertations/ 

theses 

18 

7 

Trade journal 
articles 

46 

13 

•Grant activity was described but the concept of ROI was applied to a 
different topic that was also discussed. 

•Were announcements or descriptions of funded projects that made very 
general statements. 

•One article from the Baltimore Sun was categorized by ProQuest as a trade 
journal article. It was removed from consideration since it was a newspaper 
item. 
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A simple form of content analysis (Gall, 
Gall & Borg, 2010; Neuendorf, 2002) was 
employed to consider the material 
ultimately judged to be applicable to the 
purpose of this review. Each of the articles 
was read in its entirety, with the particulars 
of the application of ROI to grants noted. 
The material captured for each source was 
the type of publication, author(s), year of 
publication, title of the article, methodology 
or evidence pattern employed, and quotes 
or summaries portraying the application of 
the concept return on investment utilized in 
the article with respect to grant activity. 
Patterns in the quotes and summaries were 
sought and descriptive themes identified. 

Results 

The sources identified as relevant 
appeared in a wide variety of publications 
and addressed a diverse set of topics. This 
indicates that interest in ROI for grants is 
not isolated to a few disciplines or areas of 
professional practice. 

All of the relevant articles in the 
"scholarly publications" set were from peer- 
reviewed journals. The 14 relevant scholarly 
articles represented 11 publications, with 
The Lancet as the most frequent publisher (3 
articles) and Implementation Science as the 
second most frequent (2 articles). Four of 
the 11 publications focused on medicine, 
with the remainder distributed across 


science disciplines (4), health (3), promotion 
of the uptake of research findings (2), and 
psychiatry (1). Three of the pieces from 
scholarly journals were editorials or opinion 
statements rather than research reports. The 
applicable dissertations were written in the 
United States and Canada, at institutions in 
five states and two provinces. The authors 
had received degrees from schools in the 
Ivy League, Big Ten, and Atlantic Coast 
Conference, from prestigious private 
research-intensive institutions, and from 
state/province universities. Dissertation 
topics addressed the fields of corporate 
culture, institutional change, federal 
funding guidelines and patterns, 
community colleges that pursue large 
grants, research administration and 
proposal development, corporate 
philanthropy, project sustainability, and 
information technology. 

The trade journal sources were 
distributed across the fields of healthcare 
(6), library science (4), education (2), 
criminal justice (1), and government (1). 
Statements of expert opinion were the most 
common form of trade journal content (6), 
followed by case studies (5), a report of 
original survey research (1), a summary of 
topics and outcomes from a national 
association meeting (1), and a report 
regarding published research (1). 
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In the Canadian Medical Association 
Journal, Frank and Nason (2009) suggested 
that there are two basic orientations to 
"proof of value-for-money" in the grant 
world. Given: 

intense interest in defining the social, 
health and economic impacts of health 


research investments globally and in 
Canada, ... 2 main approaches have 
been used over the past 20 years to 
measure return on investments....'top- 
down' econometric calculation...[and] a 
'bottom-up'...'payback' model...[which] 
has involved logic-model tracking of 
new knowledge" (Frank & Nason, 2009, 
p. 528). 


Table 3 


Topics of. Publications for, and Fields of Journal Articles 


Author(s) and Year 

Topic 

Journal 

Journal's Field(s) 

Bisias, Lo & Watkins 
(2012) 

Allocation of NIH funds 

PLoS ONE 

Primary research in science 
and medicine 

Chan et at. (2014) 

Impacts of research 

The Lancet 

Medicine 

Couee (2014) 

Economic impact of research 

EMBO Reports 

Microbiology 

Frank & Nason (2009) 

Measuring the benefits of 
research 

Canadian Medical 
Association Journal 

Medicine 

Glasgow et al. (2012) 

Improving research process 

American Journal of 

Public Health 

Public health 

Holmes, Scarrow & 
Schellenberg (2012) 

Translating evidence into 
science 

Implementation Science 

Scientific study of methods 
to promote the uptake of 
research findings 

Johnston, Rootenberg, 
Katrak, Smith & Elkins 
(2011) 

Costs and benefits to society 
of NIH clinical trials 

The Lancet 

Medicine 

Kalutkiewicz & Ehman 
(2014) 

Metrics for NIH activity 

Nature Biotechnology 

Science and business of 
biotechnology 

The Lancet (2011) 

Reasons to financially support 
the Global Fund 

The Lancet 

Medicine 

Mcllwain (2010) 

Economic return of research 

Nature 

International weekly journal 
of science 

Nicol (2008) 

Benefits derived from 
dissemination knowledge 

Health Law Journal 

Health law 

Rettig (2004) 

Suggestions for reorganizing 
the National Institutes of 

Health 

Health Affairs 

Health 

Stone & Lane (2012) 

Beneficial impacts of research 

Implementation Science 

Scientific study of methods 
to promote the uptake of 
research findings 

Weiss (2007) 

Measuring the impacts of 
medical research 

American Journal of 
Psychiatry 

Psychiatry 
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This description parallels the orientations 
described by Weiss (2007): "financial 
outcomes measures" and "impact...on the 
end user" (p. 206). These two definitions 
proved to be accurate representations of the 
content of the scholarly publications (Table 
4), that of the trade journals (Table 5), and 
that found in the dissertations. Some 
authors used both meanings in their 
presentation. There were also some 
idiosyncratic applications of ROI, such as 
Bisias, Lo, and Watkins' use as "impact on 
U.S. years of life lost" (2012, p. 1) but each 
of these fit within Frank and Nason's 
"payback" (benefit derived) category. 

The shortest article from a scholarly 
journal, an editorial board opinion 
statement, did not clearly define the 
intended meaning of return on investment. 
Eight of the 14 sources used both of the 
observed patterns of meaning for ROI 
within one article, five used just one 
definition of the phrase (one econometric 
and four benefit derived), and three of the 
14 sources were written to critique use of 
the ROI concept in evaluating grant-funded 
activity. Five of the trade journal sources 
used return on investment with respect to 
grants in a strictly financial sense, three 
applied the concept strictly as a 
consideration of benefits derived, and three 
used both meanings. Two of the trade 


journal sources did not clearly define the 
intended meaning of return on investment 
even though they connected the idea 
directly to grant activity. The dissertations 
did not depart from the usage pattern 
described by Frank and Nason (2009) and 
Weiss (2007) that has been illustrated here 
with the scholarly and trade publication 
content. 

A small but focused subset of the 
scholarly and trade journal articles 
considered the effectiveness of applying the 
concept of ROI to grant activity. These 
concerns were the primary topic of five of 
the 34 sources considered in this review, 
and formed a substantial concern in two 
other sources. The seven sources that 
critiqued application of ROI in the context 
of grant-funded activity were all opinion 
pieces, with five appearing in scholarly 
journals (Couee, 2013; Frank & Nason, 2009; 
Mcllwain, 2010; Nicol, 2008; Weiss, 2007) 
and the other two in trade journals (Corbyn, 
2009; Moriarty, 2010). 

Multiple authors suggested categories in 
which the outcomes and benefits of grant- 
funded activity could be considered. These 
categories generally described broad themes 
summarizing types of derived benefits. 
However, several authors extended their 
treatment beyond simple categorization to 
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Table 4 


Uses of the Phrase "Return on Investment 

" in the Scholarly Literature 

Author(s) 
and Date 

Topic 

Pattern of Use for "Return on Investment" 

Econometric Orientation (including authors arguing against this pattern) 

Bisias, Lo, & 

Watkins (2012) 

Proof-of-concept for a 
quantitative method of 
allocating NIH funds 

"...is analogous to managing an investment portfolio: in both cases, there are 
competing opportunities to invest limited resources" (p. 2). 

Couee (2013) 

Opinion regarding 
"procedures and standards, 
such as applying economic 
principles of quality 
assurance to research" 

(from abstract) 

"...governments and funding agencies are increasingly interested in the 
performance and cost efficiency of specific research institutes, projects, teams 
and individuals. To make these 'value-for-money' assessments, many countries 
rely on agencies and define criteria to evaluate research and carry out academic 
benchmarking" (p. 222). 

Frank & Nason 
(2009) 

Measuring social, health, 
and economic benefits of 

research 

"... intense interest in defining the social, health and economic impacts of health 
research investments globally and in Canada as proof of value-for-money" (p. 
528). 

" .. .model that "monetized improvements in life expectancy and quality of life" 

(p. 528). 

Johnston, 

Rootenberg, Katrak, 
Smith & Elkins 
(2011) 

Effect of NIH clinical trials 
on public health 

".. .at a total cost of $3.6 billion.. .net benefit to society at ten years was $15.2 
billion" (p. 1319). 

Kalutkiewicz & 
Ehman (2014) 

Metrics for NIH activity 

"... economically productive use of taxpayer dollars.... It has been estimated 
that approximately 30% of the total value of the NASDAQ has roots in academic 
research. More specifically, a 2008 study concluded that every $1 spent on NIH 
research results in $2.21 in local economic impact" (p. 536). 

Mcllwain (2010) 

Considers case for 

economic return of 

research 

"Collins has recently cited a report by Families USA, a Washington DC-based 
health advocacy group, which found that every US $lspent by the NIH typically 
generates $2.21 in additional economic output within 12 months" (p. 682). 

Nicol (2008) 

Types of return for 
dissemination of 
knowledge generated 

"...provides return on investment, potentially allowing them to 
reduce...monetary contribution" (p. 210). 

"...benefits in terms of financial return on investment" (p. 234). 

Stone & Lane (2012) 

Proposes logic model- 
based system for 
consideration of "the 

intended beneficial 
impacts" 

".. .a return on the public investment can be realized through three outcomes: 
broad and economical..., revenue from sales captured as profit..., and 
generation of new tax revenue" (p. 16). 

Weiss (2007) 

Measuring the impact of 
medical research 

"...financial outcomes measures seem removed from the core mission of 
academic medical research" (p. 206). 

Payback (Benefit Derived) Orientation 

Bisias, Lo, & 

Watkins (2012) 

Proof-of-concept for a 
quantitative method of 
allocating NIH funds 

ROI as "impact on U.S. years of life lost" (p. 1). 
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Chan et al. (2014) 

Opinion about improving 
understanding of research 
and its impact 

"Because unreported studies do not contribute to knowledge, they do not 
provide returns on the investment of research resources or the contributions of 
participants" (p. 257). 

Frank & Nason 
(2009) 

Measuring social, health, 
and economic benefits of 

research 

"'...bottom-up'...'payback model'...has involved logic-model tracking of new 
knowledge in phases from knowledge production at the researcher level.. .to 
secondary outputs and adoption to final outcomes" (p. 528). 

Glasgow et al. (2012) 

Proposes ways to improve 
progress from idea 
dissemination to 
implementation research 

"...true return on research investment requires improvements in the adoption 
and implementation of effective interventions within discrete clinical and 
community settings" (p. 1277). 

Holmes, Scarrow & 
Schellenberg (2012) 

Description of "how one 
funding agency 
determined its [knowledge 
translation] role 
and...developed a model" 

(P-1) 

".. .return on investments in health research in the form of societal and health 
system benefits" (p. 1). 

Johnston, 

Rootenberg, Katrak, 
Smith & Elkins 
(2011) 

Effect of a NIH program of 
clinical trials on public 
health and resulting costs 

"...effect of trial results on medical care and health" (p. 1319). 

Kalutkiewicz & 
Ehman (2014) 

Metrics for NIH activity 

".. .the return on investment for NIH research should be measured in terms of 
extended human life expectancy, reduced burden of disease, and long-term 
economic impact" (p. 537). 

Mcllwain (2010) 

Considers case for 

economic return of 

research 

"Biomedical research has generally been looked at for its health benefits" 

(p. 682). 

Nicol (2008) 

Types of return for 
dissemination of 
knowledge generated 

"...10% social rate of return based on building of the basic knowledge stock" (p. 
215). 

Rettig (2004) 

Discusses 

recommendations made 
regarding reorganization of 
NIH 

ROI as "improved health of the American public" (p. 257). 

Stone & Lane (2012) 

Proposes logic model- 
based system for 
consideration of "intended 
beneficial impacts" (from 
abstract) 

"...bibliometrics about research outputs (published discoveries),...transfer 
metrics about development outputs (patented prototypes)" (p. 1). 

Weiss (2007) 

Measuring the impact of 
medical research 

ROI "aligned with the amelioration of disease" (p. 206). 

"...impact of these activities on the end user...[as] measurement of outcomes" 

(p. 206). 

Unclear What Pattern is Meant 

The Lancet (2011) 

Five reasons to financially 
support the Global Fund 

Uses "largest and broadest return on investment" in undefined sense (para. 2). 
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Table 5 


Uses of the Phrase "Return on Investment" in Trade Journals 


Author(s) and 
Date 

Topic 

Pattern of use for "Return on Investment" 

Econometric Orientation (including authors arguing against this pattern) 

Anonymous 

(2011) 

Dept, of Energy reporting 
about ARRA's impact on 
the smart grid 

. .return on investment (ROI) and could help make the case for utilities to further 
in vest... [because] the savings from increased efficiency make up for the 
investment" (para. 6). 




Berry (2014) 

Transformation of 

Alberta's Edmonton Public 
Library 

"...the program allowed savings or reallocation of over $3.56 million in Canadian 
dollars.. .to Edmonton's downtown.. .with an initial investment of $630,000 

Canadian" (para. 8). 

Channing 

(2012) 

Sinai Health System's 
community health educator 
program in pediatric 
asthma 

".. .for every dollar we invest.. .we have precluded spending up to $15 in hospital 
emergency department visits or inpatient admissions - an incredible return on 
investment" (para. 10). 

Corbyn (2009) 

Discusses the absence of 
metrics in ARRA funding 

"...what level of economic return can taxpayers reasonably expect for their 
investment" (para. 7). 

Flagg (2005) 

Report on American 

Library Council meeting 

"LSTA funding.. .reached an estimated value of over $21 million - a 13-to-l return 
on investment" (para. 28). 

McCune (2007) 

Tips for grant success 

".. .funders view their financial support as an investment in your library" (para. 25). 

Moriarty (2010) 

Critique of changes in the 
British government's grant¬ 
funding patterns 

".. .prioritise research areas on the basis of their perceived impact on the economy" 
(para. 3). 

"...direct, short-term effect on the economy" (para. 9). 

STEM Cell 

Week (2007) 

Proposed changes to 
intellectual property 
regulations 

"...potential to diminish a grantee’s return on investment, increasing financial risk" 
(para. 3) 

Payback (Benefit Derived) Orientation 

Bawden et al. 
(2010) 

Evaluation of an 
emergency medicine 
national research grant 
competition 

".. .dedicated EM research funding should be continued to stimulate productivity" 

(p. 33). 

Brunell (2009) 

Report on a nursing grant 
project 

".. .maximizing use of current and preparing for future technology" 

(para. 8). 

Corbyn (2009) 

Discusses the absence of 
metrics in ARRA funding 

"quality-of-life or public-policy benefits" (para. 15). 

McCune (2007) 

Tips for grant success 

"... actual return on investment.... Although not all results 

are quantifiable, you should have some sort of evaluation to determine if your 

proposal will have made an effect or a positive change" (para. 19). 

Moriarty (2010) 

Critique of changes in the 
British government's grant¬ 
funding patterns 

"J.J. Thomson, discoverer of the electron, said....'Research in pure science is made 
without any idea of application to industrial matters, but solely with the view of 
extending our knowledge of the laws of Nature'" (para. 10). 

Raths (2009) 

Considerations related to 
seeking grants 

".. .share data points - qualitative and quantitative about return on investment, both 
in terms of patient health and improved efficiencies" (para. 27). 

Unclear What Pattern is Meant 

Gavigan (2008) 

Police communications 

Advocates seeking "grants because the return on investment (ROI) can be huge" 

(para. 16). 

McGill (2013) 

Learning taking place at 
HRSA-funded centers 

"We have to keep our ears to the community to make sure we're providing the best 
return on investment we can for the funding we do get" (p. 3). 
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suggesting systems for analysis of benefits 
derived from grant-funded research. 

Limitations 

The primary limitation applicable to this 
review is that it is the work of one person. 
Any bias he had regarding the topic could 
have influenced data-gathering, data 
analysis, and presentation of results. The 
methodology employed, incorporating 
comprehensive consideration of the 
literature available on the ProQuest 
database, pre-specified criteria, organic 
development of categories and labels, and 
frequent crosschecks of data and results, 
was chosen as a means of minimizing both 
bias and human error. 

Discussion and Conclusions 

The purely fiscal sense of return on 
investment is unlikely to disappear from the 
discussion of grants. Several of the sources 
cited in this review described governments' 
increasing emphasis on the economic 
impact of the research they fund; in the U.S. 
(Kalutkiewicz & Ehman, 2014; Mcllwain, 
2010), in Canada (Frank & Nason, 2009; 
Joosse, 2009), and in the United Kingdom 
(Corbyn, 2009; Moriarty, 2010). The issue is 
and is likely to remain, "what level of 
economic return can taxpayers reasonably 
expect for their investments" (Corbyn, 2009. 
para. 7)2 Yet the presence of two patterns of 
usage for return on investment in reference 


to grant-funded activity, "econometric" 
(Frank & Nason, 2009, p. 528) and the 
broader set of benefits derived ("impact of 
these activities;" Weiss, 2007, p. 206), brings 
an issue into focus. 

Corbyn (2009) cites Mansfield's 1991 
study, which showed $0.28 annual return in 
perpetuity for every $1 in grant funds 
distributed. Mcllwain (2010) referenced 
"a report by Families USA.. .which found 
that every US $1 spent by the NIH typically 
generates $2.21 in additional economic 
output within 12 months" (2010, p. 682). 
According to no lesser authority than Dr. 
Francis Collins, the director of the National 
Institutes of Health, "[b]iomedical research 
has generally been looked at for its health 
benefits, but the case for it generating 
economic growth is pretty compelling" (p. 
682). However, a small but noteworthy 
cohort of authors believes this perspective is 
based on a flawed application of ROI 
(Collins quoted in Mcllwain, 2010). 

That "academic research has a direct, 
short-term effect on the economy" 

(Moriarty, 2010, para. 9) is the basis of the 
objections to a solely economic perspective 
of grant ROI. Mcllwain noted that, "some 
economists question the basic assumption 
behind such models" (2010, p. 682) (i.e., 
direct linear relationships between grant 
funds invested and economic output). In 





Research Management Review, Volume 21, Number 1 (2016) 


fact, Mcllwain also stated that no one can 
assert "a certain amount of research input 
will generate corresponding economic 
outputs" (p. 682) of a known degree or 
breadth. This is the case as "[t]he economic 
impact of research is felt in myriad ways, 
both direct and indirect" (Corbyn, 2009, 
para. 20). Even should one be able to plan 
and capture measures of all direct impacts, 
preparing a plan to measure indirect or 
spillover impact implies a prescience that 
simply is not possible. A related 
consideration is the presence of 
confounding influences. It is possible for 
short-term impact to be blunted or blocked 
by circumstances external to a project such 
as institutional reallocation of resources or 
personnel, competing initiatives, and social 
or cultural pressures that affect 
participation. Long-term impacts, like the 
success or failure of a business venture 
developed from a grant-funded project, can 
also be influenced by multiple factors. A 
lack of success "could be just as much to do 
with a failed marketing campaign as the 
innovation itself" (Corbyn, 2009, para. 25), 
thereby violating the notion of direct, linear 
relationships between funding and its 
impacts. Even the abbreviated list of 
examples shared here indicates that the 
relationship of grant project or process to 


impact may not be direct and certainly is 
not linear. 

Due to the lack of linear relationships, 
the foremost argument against the 
application of fiscal ROI to grant activity is 
that it does not conform to the patterns 
necessary for the calculation. Corbyn stated 
that, "[mjeasuring the economic return on 
investment from scientific research is a 
difficult, if not impossible, task" (2009, para. 
9) and "[tjrying to put a number on it is a 
'pretty fruitless task/ particularly if quality- 
of-life or public-policy benefits are being 
fitted with pound signs" (para. 15). As 
Mcllwain (2010) noted, "[a] key 
problem.. .has been economists' inability to 
measure the costs of research as well as the 
benefits" (p. 684). This circumstance 
prevents accurate calculation of return on 
investment as one must know and take into 
account the inputs (costs) and the outcomes 
(benefits). If the factors included in the 
calculation fall short of a "full accounting" 
or are skewed based upon preference or 
perspective, "the resulting ROI figures can 
be meaningless" (Wheelen & Hunger, 2004, 
p. 258). One need only consider 
Bloomgarden's suggestion of "prestige 
generation" (2008, p. 19) or the descriptions 
of grant ROI selected from some of the 
sources for this review (Table 6) to see the 
variety of perspectives regarding grant 
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outcomes—many of the suggested in a maimer that would facilitate monetary 

outcomes would be a challenge to measure valuation and ROI calculation. 


Table 6 

Selected Phrases Used by Sources to Describe ROI 


Source 

Description Applied 

Source 

Description Applied 

Berry (2014) 

Social return 

Kalutkiewicz & Ehman 
(2014) 

Extended human life 
expectancy 

Bisias, Lo, & Watkins 
(2012) 

Years of life lost 

Reduced burden of 

disease 

Bradford (2008) 

Cultural ROI 

Long-term economic 
impact 

Corbyn (2009) 

Economic return 

Litwin (2008) 

Change in market share 

Eckert (2011) 

Indications of desired 
impact 

McCune (2007) 

Measured outcomes 

Social and business 

value 

Mcllwain (2010) 

Health benefits 

Customer loyalty 

Generating economic 
growth 

Frank and Nason (2009) 

Life expectancy 

Mills (2008) 

Tangible products 

Quality of life 

Moriarty (2010) 

Direct, short-term effect 
on the economy 

Gavigan (2008) 

Benefit derived 

Chan et al. (2014); 
Moriarty (2010) 

Extending knowledge 

Glasgow et al. (2012) 

Societal benefits 

Raths (2009) 

Data regarding patient 
health 

Health system benefits 

Data regarding 
improved efficiencies 

Johnston, Rootenberg, 
Katrak, Smith & Elkins 
(2011) 

Effects on medical care 

Rettig (2004) 

Improved health of the 
U.S. public 

Effects on health 

Weiss (2007) 

Amelioration of disease 


Nicol (2008) noted the possibility of a 
variance of perspectives regarding a single 
project, saying that individuals with 
differing sets of responsibilities and 
backgrounds are "likely to bring quite 
different interests and experience to 
the...table. University administrators are 
likely to focus on cost recovery whereas the 


driver for industry is product development" 
(Nicol, 2008. p. 212). This can produce a 
"clash of cultures" (ibid.) around what 
constitutes acceptable and desirable 
outcomes and measures. Litwin (2008) 
addressed this issue clearly in his 
dissertation: "If there is no generally 
accepted method of measuring 
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success.. .then return on investment in 
research capacity cannot be measured" (p. 
93). 

It is a practical impossibility to track and 
assign a monetary value to all direct and 
indirect inputs and outcomes that occur in 
grant-related activity, from proposal 
production to project implementation and 
management and on to long-term impacts 
(both anticipated and unanticipated). As a 
result, values calculated for grant ROI are 
speculative or context-limited at best and 
may even be spurious. This conclusion has 
been reached by multiple parties. Frank and 
Nason (2009) directly stated that no 
validated method for measuring return on 
investments for grants exists. Stone and 
Lane (2012) concurred, noting "the absence 
of comprehensive models and metrics" (p. 
1), as do Corbyn (2009), Couee (2013), 
Mcllwain (2010), Moriarty (2010), and Nicol 
(2008). 

The second line of argument against 
calculation of fiscal ROI for grant-funded 
activity is the presence of erroneous 
assumptions. As has just been noted, "there 
is no consensus about what constitutes 
impact" (Corbyn, 2009, para. 9; see also 
Table 6), nor is there consensus about which 
potential impacts should be considered for 
various types of projects. Thus, the 
assumption that grant impacts are known 


and can be consistently measured is flawed. 
Without common understanding of what 
"impact" entails and how to capture a 
reliable measurement of it, even the most 
meticulous of calculations become limited 
and overly context-specific. 

The application of fiscal ROI to grant- 
funded projects assumes simple, uniform 
measurement. Yet the measurements taken 
cannot be simple, as they address complex 
and sometimes conflicting constructs (Table 
6). The complication of comparing ROI 
across research contexts and projects is an 
apt illustration. As a UK expert, Ben Martin 
of the Institute of Science and Technology 
Policy Research at the University of Sussex, 
has asked, "'How do you weigh up Pounds 
1 billion of spin-off increases against a 
potential policy outcome of 100,000 fewer 
people becoming obese'" (Martin, quoted in 
Corbyn, 2009, para. 40)? In addition, the 
measures of process and products in grant 
activity are not uniform. Research practice 
is dynamic, with new developments 
occurring annually and with a variety of 
tools and applications available to 
accomplish similar purposes. While 
scientific enquiry dictates consistency in 
process within an investigation or 
intervention, it is possible for advancements 
and alterations to occur between funding 
cycles. The support facilities, research 
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venues, and project personnel may change 
for a single continuous project from one 
funding cycle to the next, introducing 
variations in resource inputs as well as in 
expertise and insight. Lead investigators 
and their colleagues continue to learn and 
to be influenced by the work of others as it 
becomes available, altering their 
understanding and perspectives. These 
phenomena call into question the 
assumption that ROI calculations completed 
even three years ago will represent the same 
set of inputs and behaviors as calculations 
for the same project or similar projects at 
present. Even changes in funder reporting 
standards can impact practice in currently 
funded projects. Corbyn (2009) provided an 
example of rapid change of this type. When 
the British government "made the number 
of spin-offs a key indicator of the success of 
knowledge-transfer activities .... Numbers 
increased nearly threefold" (para. 27) in one 
year. But that '"did not mean that [the 
researchers] suddenly became three times 
better at technology transfer and generating 
economic impact.. .it meant.. .that once it 
became a performance indicator, everybody 
started maximising their score'" (Corbyn, 
2009, para. 28). Variation and malleability of 
the kind described above decidedly limits 
the ability to argue for simple, uniform 
measurement with respect to grant projects 


undermining the applicability of fiscal ROI 
calculation. 

Comparing calculated ROI between 
projects or programs assumes that the 
inputs, processes, and outcomes are 
transferable from setting to setting. Yet 
"[l]ittle is known about.. .the extent to 
which the benefits of research done in one 
country or region are specific to that area" 
(Mcllwain, 2010, p. 684). Ignoring this 
principle when seeking to summarize 
impact contradicts what is considered by 
funders to be best practice. Funding 
organizations do not generally assume 
uniformity from one setting to the next. 

This is the reason for inclusion of proof of 
replicability in proposals and for the 
funding of validation, replication, and 
dissemination studies. 

Another flawed assumption in applying 
a fiscal concept of ROI to grant activity is 
the ability to establish appropriate periods 
in which to measure impacts, especially in 
respect to research outcomes. There can be 
extended waits before a research product 
has a discernable impact (Kalutkiewicz & 
Ehman, 2014; Nicol, 2008). Indeed, "[l]ittle is 
known about how long the economic 
benefits of research take to accrue" 
(Mcllwain, 2010, p. 684). Since little is 
known about the period of time necessary 
for economic benefit to be derived, any 
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pattern utilized becomes a best guess. 
Chance or ancillary outcomes are a related 
concern. It is not uncommon for grant- 
funded projects of all types to have 
unexpected outputs or outcomes. Many 
research discoveries from "antibiotics to 
nuclear magnetic resonance have had huge 
economic impact, but were both 
serendipitous and did not hit pay dirt for 
many years" (Corbyn, 2009, para. 23). 

Those citing well-known reports and 
studies in support of calculating fiscal ROI 
for grants appear to ignore the 
shortcomings of the reports they cite. Dr. 
Benjamin Martin professor of science and 
technology studies at the University of 
Sussex describes the classic example, 
Mansfield's 1991 analysis, as "too good to be 
true" indicating it "rested on 'heroic 
assumptions' and 'flawed methodology'" 
(Corbyn, 2009, para. 17). Mcllwain (2010) 
agrees: Mansfield's data was developed "by 
interviewing chief executive officers, asking 
them what proportion of their companies' 
innovation was derived from university 
research work" (Mcllwain, 2010, p. 683). It 
is essentially a summary estimate based on 
individual estimates provided by a limited 
convenience sample, 76 CEOs, "who were 
pushed by the economist to give answers" 
(Corbyn, 2009, para. 17) in an arena of 
activity about which they were likely to be 


favorably biased. Mansfield's work is not an 
isolated instance. Robert Topel of the 
University of Chicago observes, "'It is very 
hard to take changes in public health and 
attribute their cause'" (Mcllwain, 2010, p. 
684). Yet this is the overt purpose of 
frequently cited reporting regarding ROI of 
funded research. Several examples are 
Exceptional returns: The economic value of 
America's investment in medical research and 
Rising above the gathering storm (National 
Academies, 2007), publications compiled by 
lobbyists and government-funded 
committees. Mcllwain's view is that the 
"economic benefits of research have been 
extrapolated from a small number of 
studies, many of which were undertaken 
with the explicit aim of building support for 
research investment, rather than being 
objective assessments" (Mcllwain, 2010, p. 
682). The American (Kalutkiewicz & 

Ehman, 2014; Mcllwain, 2010), Canadian 
(Frank & Nason, 2009; Joosse, 2009), and 
British governments (Corbyn, 2009; 
Moriarty, 2010) are actively gathering 
information in these areas and it is hoped 
that they can improve on the current state 
of affairs. 

It is naturally important to employ some 
form of measurement for the eventual 
benefits of grant-funded activity. Weiss 
(2007) proposed three reasons for this: 
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(1) "measuring outcomes provides a clear 
and meaningful message regarding the 
return on investment to the major funders" 
(p. 207); (2) outcomes measurement "serves 
as a compass to keep our research efforts, 
individually and collectively, on track" 
(ibid.); and (3) this measurement "makes 
explicit the societal good embedded 
in...research" (ibid.). However, "[t]he 
absence of comprehensive models and 
metrics skews evidence gathering" (Stone & 
Lane, 2012, p. 1) toward the proximal and 
the easily chronicled. This becomes 
"problematic.. .as most measurable 
socioeconomic benefits" (ibid.) occur 
through "commercial innovation" (ibid.), 
and, as a result, are very long-term 
measures without readily predictable paths. 

Even with respect to socioeconomic 
benefits, there will be a wide variety of 
possible measures. Dr. Benjamin Martin of 
the University of Sussex believed that "you 
can capture economic impact, but.. .need a 
'vast array' of indicators - about 65" 

(Martin, quoted in Corbyn, 2009, para. 26). 
This matches the system described by Frank 
and Nason *(2009) which includes "a 
starting menu of 66 preferred indicators 
designed to answer a number of potential 
impact questions" (p. 532) and the potential 
for development of other indicators. Nicol 
(2008) agreed that there are a variety of 


valid metrics, resulting in what is unlikely 
to be a simple system of measurement or 
one that is chiefly focused on fiscal ROI. 

Weiss' statement of reasons to compile 
ROI information, referring to both the fiscal 
outcomes and benefits-derived definitions 
of ROI, summarizes the need for clear and 
balanced activity in this area. Having a 
means of chronicling "the outcomes 
of.. .work would provide.. .funders with 
clearer mission-central return-on- 
investment feedback, would make explicit 
the benefits of science to an increasingly 
skeptical public, and would serve as a... 
guide the scientific community" (Weiss, 
2007, p. 206). 

Recommendations 

Consideration of return on investment 
has a place in the grant world, even though 
fiscal ROI should be viewed as a tool that is 
limited in accuracy, scope, and 
applicability. Therefore, it is incumbent on 
grant professionals to utilize the most 
appropriate patterns and understandings 
for assessment of the benefits of grant- 
funded activity, rather than the most 
convenient, and to advocate for the most 
appropriate patterns with administrators, 
government officials, faculty, and peers so 
that these patterns become and remain the 
"best practice" in research administration. 
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Toward this end, the benefits derived 
perspective on ROI should become the 
preferred perspective in a grants context. 
This is the view of the Research Councils of 
the United Kingdom, which suggested "a 
broad approach to describing the impact of 
research" (Research Councils of the United 
Kingdom, quoted in Corbyn, 2009, para. 35). 
The goal is to look "at economic effects, 
[but]...also consider social benefits such as 
improved public-policy and quality-of-life 
outcomes" (Corbyn, 2009, para. 15) by 
integrating case studies in order to "achieve 
an 'ever more persuasive' evidence base for 
the impact of research council funding" 
(Corbyn, 2009, para. 38). Methodology in 
gathering qualitative accounts and in 
making generalizations from these accounts 
becomes a concern in this approach, but a 
purely quantitative fiscal mode of 
consideration poses equal, if not greater, 
concerns and limitations, as argued above. 

In order to forego a "clash of cultures" 
(Nicol, 2008, p. 212) around what 
constitutes acceptable and desirable 
measures, it would be wise to follow the 
focus on the purpose behind measurement 
stressed by Weiss (2007), Moriarty (2010), 
and Couee (2013). Put simply, assessment 
will be ineffective without an identified 
purpose for the measurement and analysis 
conducted. Thonon et al. (2015) provided a 


strong example of identified purpose of 
assessment: "[according to the Canadian 
Academy of Health Sciences, evaluation of 
research is carried out for three main 
purposes: accountability purposes, 
advocacy purposes, and learning purposes" 
(p. 2). Several authors of dissertations 
suggested different patterns for organizing 
the basic purposes of grants: Mills (2008) 
suggested that institutions of higher 
education (IHE) assess tangible products in 
three areas, "core mission areas of 
instruction, research, and public service" (p. 
30) while Eckert (2011) suggested that 
corporate funders are attuned to achieving 
"both social and business value" (p. 71). 
Nicol (2008) noted that each stakeholder or 
group is likely to have "quite different 
interests and experience" (p. 211), which 
will influence their perspective of the 
benefits to be considered. The most direct 
way to resolve differences of this type is to 
clearly identify the entity's purposes for the 
enterprise and seek to measure according to 
these purposes. 

As highlighted in Nicol (2008), different 
purposes for grant-funded activity exist 
with respect to each stakeholder and 
organization, specific undertakings, and 
each funder. In several countries, funding 
agencies supported by the national 
government are actively engaged in the 
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development or early implementation of 
reporting systems that seek to capture more 
and more relevant data related to projects 
they fund. However, this does not resolve 
the matter for institutions of higher 
education, non-profits, healthcare entities, 
or other organizations receiving grants 
specific to their own unique interests. 
Funding-program-level or even national- 
level summation of impacts does not 
address institutional, local, partnering 
organization, or individual outputs and 
outcomes, unless very few projects are 
being funded at the national level. IHEs and 
other funded parties must resolve this 
challenge for and within their own 
purposes and setting. Two immediately 
accessible approaches can be taken, or even 
combined, to meet this need. These are an 
institutional values-specific benefit analysis 
process and a balanced scorecard approach. 

Uttam and Venugopal (2008) developed 
a quantitative benefit analysis system that 
takes institutional purposes into account. 
While the system was devised in India for a 
research laboratory, its basic structure and 
essential elements are easily adaptable for 
organizations that receive grant funding in 
any part of the world. Uttam and 
Venugopal's system plots points on a four- 
quadrant graph by using values from a 
"contract value index" (p. 60) and a 


"potential benefits index" (p. 59). The 
contract value index is a simplified ROI 
calculation, the "ratio of the actual contract 
fee realized from a [project] to the 
prevailing man-year rates of the 
organization" (p. 60). The man-year rate 
comprises all known organizational costs 
for the program, staff time, direct costs, and 
indirect cost, calculated as a yearly average. 
While as described above, this is likely 
inadequate for an accurate ROI calculation, 
utilizing this pattern for all projects will 
allow comparison of similar sets of 
summary data (known costs). Potential 
benefits ratings result from consideration of 
a ten-item list that can be weighted in favor 
of primary purposes while still considering 
other goals. In the process, organizations 
rate projects according to known 
institutional priorities. Both scores are then 
plotted on a four-quadrant graph allowing 
comparison of projects. To aid with 
comparison and characterization of projects, 
categorical descriptions for each quadrant 
of the graph were created ("move away," 
"futuristic," "desirable," and "beneficial" 
[Uttam & Venugopal, 2008, p. 65]). Uttam 
and Venugopal completed a proof-of- 
concept study with historical data from 
their organization and used the data from 
that pilot for their 2008 publication. Their 
system allows quantitative comparison of 



Research Management Review, Volume 21, Number 1 (2016) 


projects and aids in determining the relative 
value of each project to the organization, 
from nominal value without the potential 
for long-term impact ("move away") to a 
"high level of.. .long-term benefits 
and.. .substantial revenues upfront" 
("desirable") (Uttam & Venugopal, 2008, p. 
65). 

Another immediately applicable 
assessment system can be used in 
conjunction with Uttam and Venugopal's 


pattern or on its own. This is rubric-based 
evaluation or a "balanced scorecard," a 
pattern long employed in business settings 
(Wheelen & Hunger, 2004). This system 
includes "nonfinancial as well as financial 
measures" (p. 250), requires measurable 
objectives (e.g., SMART goals) and key 
performance indicators for each objective, 
and as a result is customizable, with 
institutional goals directing the formation of 
the scorecard. 


Table 7 


Illustration of Project Purposes Divided Categorically 


Spheres of Measurement 

Possible Considerations 

Fiscal 

Fiscal purposes for grants are limited to input concerns like volume, 
resourcing, and consistency (e.g., gradually decreasing institutional dollars 
supporting the project, provision of all equipment required to initiate the 
project) and results in two categories: outputs (e.g., licensable product 
developed) and outcomes (e.g., spin-off company established). 

Stakeholder 

A variety of stakeholders exist who may have identifiable purposes related to 
a grant-funded project. Examples are the funder, the awardee 
organization, the research team, the entity of which the awardee 
organization is a member (e.g., state university system office), student 
workers, participants, and the public. 

Innovation and learning 

Innovation and learning purposes are limited to the behavioral (e.g., process 
initiated, contacts made, counseling provided) and result spheres (e.g. 
increased performance, completion of a credential). 

Impact 

Impact purposes can occur in respect to the setting (e.g., improved process, 
facilitating multiple projects through acquisition of a large item of 
equipment), the field of the investigation (e.g., new method or knowledge), 
or society (e.g., improved healthcare delivery). 


To employ a balanced scorecard system, 
the first topic to consider is the purpose(s) 
for the project or initiative. These can be 
classified in four categories: fiscal, 
stakeholder, innovation and learning, and 


impact. It should be noted that a project 
may have more than one purpose in a 
category or purposes in several of the 
categories. A SMART goal should be 
written for each identified purpose. To 
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establish the key performance indicators for 
the SMART goal(s), a series of questions are 
asked starting with identification of the 
purpose. These are: (1) What is the purpose 
(or what are the purposes) for undertaking 
this project and the associated standard(s) 
by which success will be measured?; (2) Is 
this measurement a process (chronicling 
activity) or product (chronicling results) 
consideration?; (3) Is this a fiscal, 
stakeholder, innovation and learning, or 
impact measure?; (4) Does the measure 
require monitoring of inputs, activities, or 
results?; (5) If a result is in view, is this an 
output or an outcome concern?; (6) What 
data will be required for the proposed 
measurement, and are they qualitative, 
quantitative, or are both needed?; and (7) 
How will the data be processed or 
analyzed? Once these questions have been 


answered for each goal, a simple table can 
be constructed listing the project objectives 
(SMART goals) in the left-hand column and 
the performance indicator (the standard 
against which performance will be 
compared) in a middle column, leaving 
room in the right-hand column for the 
material generated by completing the 
measurement. The result is a clearly defined 
rubric for assessing performance in a 
project. In this system, it is only possible to 
compare performance data between projects 
should they have exactly the same set of 
purposes and performance measures, which 
is unlikely. However, the system does make 
it possible to summarize project results that 
are associated with various institutional 
purposes and note the achievements in each 
area. 


Table 8 

Example of a Balanced Scorecard for a Grant Project 


SMART Objective Performance Indicator 

Measure 

Fiscal Purposes 

1. SMART Objective for the first fiscal measure. 

Baseline or standard 

Measure taken 

2. SMART Objective for the second fiscal measure. 

Baseline or standard 

Measure taken 

Stakeholder Purposes 

1. SMART Objective for the first stakeholder measure. 

Baseline or standard 

Measure taken 

2. SMART Objective for the second stakeholder measure. 

Baseline or standard 

Measure taken 

Innovation and Learning Purposes 

1. SMART Objective for the first innovation and learning 

measure. 

Baseline or standard 

Measure taken 

2. SMART Objective for the second innovation and learning 

measure. 

Baseline or standard 

Measure taken 

Impact Purposes 

1. SMART Objective for the first impact measure. 

Baseline or standard 

Measure taken 

2. SMART Objective for the second impact measure. 

Baseline or standard 

Measure taken 
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Organizations receiving grant funding 
are called upon by the funder, the 


thoughtfully prepared and applied system 
is a must for completing this process. While 
this task is not as simple as some would like 
for it to be, consistent application of well- 
constructed approaches can produce 
meaningful results. 


organization's internal and external 


stakeholders, and the public to demonstrate 
responsible use of resources, including 


noting benefits derived from the projects. A 
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